Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation

Existing neural network-based autonomous systems are shown to be vulnerable against adversarial attacks, therefore sophisticated evaluation of their robustness is great importance. However, evaluating the under worst-case scenarios based on known attacks not comprehensive, mention that some them even rarely occur in real world. Also, distribution safety-critical data usually multimodal, while most traditional and methods focus a single modality. To solve above challenges, we propose flow-based multimodal scenario generator for decision-making algorithms. The proposed generative model optimized with weighted likelihood maximization gradient-based sampling procedure integrated improve efficiency. generated by efficiently querying task algorithms simulator. Experiments self-driving demonstrate our advantages terms testing efficiency modeling capability. We evaluate six Reinforcement Learning traffic provide empirical conclusions about robustness.